The present invention relates generally to the fields of sensing, detecting, monitoring and/or identifying various objects, and parts thereof, which are located within the passenger compartment of a motor vehicle. The invention also relates to methods for controlling deployment of an occupant restraint in a vehicle.
Further, the present invention relates to an efficient and highly reliable method for developing a system for detecting the orientation of an object in the passenger compartment, e.g., a rear facing child seat (RFCS) situated in the passenger compartment in a location where it may interact with a deploying occupant protection apparatus, such as an airbag, and/or for detecting an out-of-position occupant. The resulting system permits the control and selective suppression of deployment of the occupant protection apparatus when the deployment may result in greater injury to the occupant than the crash forces themselves. This is accomplished in part through a method of determining the placement of transducers of the system, a method of developing a pattern recognition system including a method of training a neural network and/or a method for developing a system for the novel analysis of the signals from the transducers.
The application of the occupant position sensor to a new automobile vehicle model is called applications engineering. Applications engineering of occupant sensors comprises, inter alia, determining the location of the transducers, designing the transducer holders, determining the wiring layout, performing a tolerance study on the transducer locations and angular orientation, designing the circuits for the particular vehicle model, interfacing or integrating the circuits into the vehicle electronic system, and adapting the occupant sensor system to the particular vehicle model.
All of the above aspects of application engineering, with the exception of the system adaptation, are standard processes that do not differ significantly from the application engineering of any electronic system to a new vehicle model. The system adaptation, however, is unique in that it requires considerable skill and expertise and the use of novel technologies to create a system that is optimized for a particular vehicle.
1. Prior Art on Sensing of Out-of-position Occupants and Rear Facing Child Seats
Whereas thousands of lives have been saved by airbags, a large number of people have also been injured, some seriously, by the deploying airbag, and thus significant improvements to the airbag system arc necessary. As discussed in detail in one or more of the patents and patent applications cross-referenced above, for a variety of reasons, vehicle occupants may be too close to the airbag before it deploys and can be seriously injured or killed as a result of any deployment thereof. Also, a child in a rear facing child seat which is placed on the right front passenger seat is in danger of being seriously injured if the passenger airbag deploys. For these reasons and, as first publicly disclosed in Breed, D. S. xe2x80x9cHow Airbags Workxe2x80x9d presented at the International Conference on Seatbelts and Airbags in 1993, in Canada, occupant position sensing and rear facing child seat detection is required in order to minimize the damages caused by deploying airbags. It is also be required in order to minimize the damage caused by the deployment of other types of occupant protection and/or restraint devices which might be installed in the vehicle.
Initially, these systems will solve the out-of-position occupant and the rear facing child seat problems related to current airbag systems and prevent unneeded and unwanted airbag deployments when a front seat is unoccupied. However, airbags are now under development to protect rear seat occupants in vehicle crashes and all occupants in side impacts. A system is therefore needed to detect the presence of occupants, determine if they are out-of-position (defined below) and to identify the presence of a rear facing child seat in the rear seat. Future automobiles are expected to have eight or more airbags as protection is sought for rear seat occupants and from side impacts. In addition to eliminating the disturbance and possible harm of unneccessary airbag deployments, the cost of replacing these airbags will be excessive if they all deploy in an accident needlessly.
Inflators now exist which will adjust the amount of gas flowing to or from the airbag to account for the size and position of the occupant and for the severity of the accident. The vehicle identification and monitoring system (VIMS) discussed in U.S. Pat. No. 5,829,782, and U.S. patent application Ser. No. 08/798,029 filed Feb. 6, 1997 among others, will control such inflators based on the presence and position of vehicle occupants or of a rear facing child seat. The instant invention is concerned with the process of adapting the vehicle interior monitoring systems to a particular vehicle model and achieving a high system accuracy and reliability as discussed in greater detail below.
The automatic adjustment of the deployment rate of the airbag based on occupant identification and position and on crash severity has been termed xe2x80x9csmart airbagsxe2x80x9d. Central to the development of smart airbags is the occupant identification and position determination systems described in the above-referenced patents and patent applications and to the methods described herein for adapting those systems to a particular vehicle model. To complete the development of smart airbags, an anticipatory crash detecting system such as disclosed in U.S. patent application Ser. No. 08/247,760 filed May 23, 1994 is also desirable. Prior to the implementation of anticipatory crash sensing, the use of a neural network smart crash sensor which identifies the type of crash and thus its severity based on the early part of the crash acceleration signature should be developed and thereafter implemented. U.S. Pat. No. 5,684,701 (Breed) describes a crash sensor based on neural networks. This crash sensor, as with all other crash sensors, determines whether or not the crash is of sufficient severity to require deployment of the airbag and, if so, initiates the deployment. A neural network based on a smart airbag crash sensor could also be designed to identify the crash and categorize it with regard to severity thus permitting the airbag deployment to be matched not only to the characteristics and position of the occupant but also the severity and timing of the crash itself (this being, described in U.S. patent application Ser. No. 08/798,029 referenced above).
The need for an occupant out-of-position sensor has also been observed by others and several methods have been described in certain U.S. patents for determining the position of an occupant of a motor vehicle. However, no patents have been found that describe the methods of adapting such sensors to a particular vehicle model to obtain high system accuracy. Each of these systems will be discussed below and have significant limitations.
In White et al. (U.S. Pat. No. 5,071,160), for example, a single acoustic sensor and detector is described and, as illustrated, is mounted lower than the steering wheel. White et al. correctly perceive that such a sensor could be defeated, and the airbag falsely deployed, by an occupant adjusting the control knobs on the radio and thus they suggest the use of a plurality of such sensors. White et al. does not disclose where the such sensors would be mounted, other than on the instrument panel below the steering wheel, or how they would be combined to uniquely monitor particular locations in the passenger compartment and to identify the object(s) occupying those locations. The adaptation process to vehicles is not described.
Mattes et al. (U.S. Pat. No. 5,118,134) describe a variety of methods for measuring the change in position of an occupant including ultrasonic, active or passive infrared radiation and microwave radar sensors, and an electric eye. The use of these sensors is to measure the change in position of an occupant during a crash and they use that information to assess the severity of the crash and thereby decide whether or not to deploy the airbag. They are thus using the occupant motion as a crash sensor. No mention is made of determining the out-of-position status of the occupant or of any of the other features of occupant monitoring as disclosed in the above-referenced patents and/or patent applications. It is interesting to note that nowhere does Mattes et al. discuss how to use a combination of ultrasonic sensors/transmitters to identify the presence of a human occupant and then to find his/her location in the passenger compartment.
The object of an occupant out-of-position sensor is to determine the location of, e.g., the head and/or chest of the vehicle occupant in the passenger compartment to enable the location of the head and/or chest to be determined relative to the occupant protection apparatus, e.g., airbag, since it is the impact of either the head or chest with the deploying airbag which can result in serious injuries. Both White et al. and Mattes et al. disclose only lower mounting locations of their sensors which are mounted in front of the occupant such as on the dashboard or below the steering wheel. Both such mounting locations are particularly prone to detection errors due to positioning of the occupant""s hands, arms and legs. This would require at least three, and preferably more, such sensors and detectors and an appropriate logic circuitry which ignores readings from some sensors if such readings are inconsistent with others, for the case, for example, where the driver""s arms are the closest objects to two of the sensors. The determination of the proper transducer mounting locations, aiming and field angles for a particular vehicle model are not disclosed in either White et al. or Mattes et al. and are part of the vehicle model adaptation process described herein.
White et al. also describe the use of error correction circuitry, without defining or illustrating the circuitry, to differentiate between the velocity of one of the occupant""s hands, as in the case where he/she is adjusting the knob on the radio, and the remainder of the occupant. Three ultrasonic sensors of the type disclosed by White et al. might, in some cases, accomplish this differentiation if two of them indicated that the occupant was not moving while the third was indicating that he or she was moving. Such a combination, however, would not differentiate between an occupant with both hands and arms in the path of the ultrasonic transmitter at such a location that they were blocking a substantial view of the occupant""s head or chest. Since the sizes and driving positions of occupants are extremely varied, trained pattern recognition systems, such as neural networks, are required when a clear view of the occupant, unimpeded by his/her extremities, cannot be guaranteed. White et al. do not suggest the use of such neural networks.
Fujita et al., in U.S. Pat. No. 5,074,583, describe another method of determining the position of the occupant but do not use this information to control and suppress deployment of an airbag if the occupant is out-of-position, or if a rear facing child seat is present. In fact, the closer that the occupant gets to the airbag, the faster the inflation rate of the airbag is according to the Fujita et al. patent, which thereby increases the possibility of injuring the occupant. Fujita et al. do not measure the occupant directly but instead determine his or her position indirectly from measurements of the seat position and the vertical size of the occupant relative to the seat. This occupant height is determined using an ultrasonic displacement sensor mounted directly above the occupant""s head.
It is important to note that in all cases in the above-cited prior art, except those assigned to the current assignee of the instant invention, no mention is made of the method of determining transducer location, deriving the algorithms or other system parameters that allow the system to accurately identify and locate an object in the vehicle. In contrast, in one implementation of the instant invention, the return ultrasonic echo pattern over several milliseconds corresponding to the entire portion of the passenger compartment volume of interest is analyzed from multiple transducers and sometimes combined with the output from other transducers, providing distance information to many points on the items occupying the passenger compartment.
Many of the teachings of this invention are based on pattern recognition technologies as taught in numerous textbooks and technical papers. Central to the diagnostic teachings of this invention is the manner in which the diagnostic module determines a normal pattern from an abnormal pattern and the manner in which it decides what data to use from the vast amount of data available. This is accomplished using pattern recognition technologies, such as artificial neural networks, and training. The theory of neural networks including many examples can be found in several books on the subject including: Techniques And Application Of Neural Networks, edited by Taylor, M. and Lisboa, P., Ellis Horwood, West Sussex, England, 1993, Naturally Intelligent Systems, by Caudill, M. and Butler, C., MIT Press, Cambridge Mass., 1990, J. M. Zaruda, Introduction to Artificial Neural Systems, West publishing Co., N.Y. 1992 and, Digital Neural Networks, by Kung, S. Y., PTR Prentice Hall, Englewood Cliffs, N.J. 1993, Eberhart, R., Simpson, P. and Dobbins, R., Computational Intelligence PC Tools, Academic Press, Inc., 1996, Orlando, Fla., all of which are included herein by reference. The neural network pattern recognition technology is one of the most developed of pattern recognition technologies.
2. Definitions
The use of pattern recognition, or more particularly how it is used, is central to the instant invention. In the above-cited prior art, except in that assigned to the current assignee of the instant invention, pattern recognition which is based on training, as exemplified through the use of neural networks, is not mentioned for use in monitoring the interior passenger compartment or exterior environments of the vehicle. Thus, the methods used to adapt such systems to a vehicle are also not mentioned.
xe2x80x9cPattern recognitionxe2x80x9d as used herein will generally mean any system which processes a signal that is generated by an object (e.g., representative of a pattern of returned or received impulses, waves or other physical property specific to and/or characteristic of and/or representative of that object) or is modified by interacting with an object, in order to determine to which one of a set of classes that the object belongs. Such a system might determine only that the object is or is not a member of one specified class, or it might attempt to assign the object to one of a larger set of specified classes, or find that it is not a member of any of the classes in the set. The signals processed are generally a series of electrical signals coming from transducers that are sensitive to acoustic (ultrasonic) or electromagnetic radiation (e.g., visible light or infrared radiation), although other sources of information are frequently included.
A trainable or a trained pattern recognition system as used herein generally means a pattern recognition system which is taught to recognize various patterns constituted within the signals by subjecting the system to a variety of examples. The most successful such system is the neural network. Thus, to generate the pattern recognition algorithm, test data is first obtained which constitutes a plurality of sets of returned waves, or wave patterns, from an object (or from the space in which the object will be situated in the passenger compartment, i.e., the space above the seat) and an indication of the identify of that object, (e.g., a number of different objects are tested to obtain the unique wave patterns from each object). As such, the algorithm is generated, and stored in a computer processor, and which can later be applied to provide the identity of an object based on the wave pattern being received during use by a receiver connected to the processor and other information. For the purposes here, the identity of an object sometimes applies to not only the object itself but also to its location and/or orientation in the passenger compartment. For example, a rear facing child seat is a different object than a forward facing child seat and an out-of-position adult is a different object than a normally seated adult.
To xe2x80x9cidentifyxe2x80x9d as used herein will generally mean to determine that the object belongs to a particular set or class. The class may be one containing, for example, all rear facing child seats, one containing all human occupants, or all human occupants not sitting in a rear facing child seat depending on the purpose of the system. In the case where a particular person is to be recognized, the set or class will contain only a single element, i.e., the person to be recognized.
An xe2x80x9coccupying itemxe2x80x9d of a seat may be a living occupant such as a human or a dog, another living organism such as a plant, or an inanimate object such as a box or bag of groceries or an empty child seat.
xe2x80x9cOut-of-positionxe2x80x9d as used for an occupant will generally means that the occupant, either the driver or a passenger, is sufficiently close to the occupant protection apparatus (airbag) prior to deployment that he or she is likely to be more seriously injured by the deployment event itself than by the accident. It may also mean that the occupant is not positioned appropriately in order to attain beneficial, restraining effects of the deployment of the airbag. As for the occupant being too close to the airbag, this typically occurs when the occupant""s head or chest is closer than some distance such as about 5 inches from the deployment door of the airbag module. The actual distance value where airbag deployment should be suppressed depends on the design of the airbag module and is typically farther for the passenger airbag than for the driver airbag.
xe2x80x9cTransducerxe2x80x9d as used herein will generally mean the combination of a transmitter and a receiver. In come cases, the same device will serve both as the transmitter and receiver while in others two separate devices adjacent to each other will be used. In some cases, a transmitter is not used and in such cases transducer will mean only a receiver. Transducers include, for example, capacitive, inductive, ultrasonic, electromagnetic (antenna, CCD, CMOS arrays), weight measuring or sensing devices.
xe2x80x9cAdaptationxe2x80x9d as used here represents the method by which a particular occupant sensing system is designed and arranged for a particular vehicle model. It includes such things as the process by which the number, kind and location of various transducers is determined. For pattern recognition systems, it includes the process by which the pattern recognition system is taught to recognize the desired patterns. In this connection, it will usually include (1) the method of training, (2) the makeup of the databases used for training, testing and validating the particular system, or, in the case of a neural network, the particular network architecture chosen, (3) the process by which environmental influences are incorporated into the system, and (4) any process for determining the pre-processing of the data or the post processing of the results of the pattern recognition system. The above list is illustrative and not exhaustive. Basically, adaptation includes all of the steps that are undertaken to adapt transducers and other sources of information to a particular vehicle to create the system which accurately identifies and determines the location of an occupant or other object in a vehicle.
In the description herein on anticipatory sensing, the term xe2x80x9capproachingxe2x80x9d when used in connection with the mention of an object or vehicle approaching another will generally mean the relative motion of the object toward the vehicle having the anticipatory sensor system. Thus, in a side impact with a tree, the tree will be considered as approaching the side of the vehicle and impacting the vehicle. In other words, the coordinate system used in general will be a coordinate system residing in the target vehicle. The xe2x80x9ctargetxe2x80x9d vehicle is the vehicle which is being impacted. This convention permits a general description to cover all of the cases such as where (i) a moving vehicle impacts into the side of a stationary vehicle, (ii) where both vehicles are moving when they impact, or (iii) where a vehicle is moving sideways into a stationary vehicle, tree or wall.
3. Pattern Recognition Prior Art
Japanese Patent 3-42337 (A) to Ueno describes a device for detecting the driving condition of a vehicle driver comprising a light emitter for irradiating the face of the driver and a means for picking up the image of the driver and storing it for later analysis. Means are provided for locating the eyes of the driver and then the irises of the eyes and then determining if the driver is looking to the side or sleeping. Ueno determines the state of the eyes of the occupant rather than determining the location of the eyes relative to the other parts of the vehicle passenger compartment. Such a system can be defeated if the driver is wearing glasses, particularly sunglasses, or another optical device which obstructs a clear view of his/her eyes. Pattern recognition technologies such as neural networks are not used. The method of finding the eyes is described but not a method of adapting the system to a particular vehicle model.
U.S. Pat. No. 5,008,946 to Ando uses a complicated set of rules to isolate the eyes and mouth of a driver and uses this information to permit the driver to control the radio, for example, or other systems within the vehicle by moving his eyes and/or mouth. Ando uses natural light and illuminates only the head of the driver. He also makes no use of trainable pattern recognition systems such as neural networks, nor is there any attempt to identify the contents of the vehicle nor of their location relative to the vehicle passenger compartment. Rather, Ando is limited to control of vehicle devices by responding to motion of the driver""s mouth and eyes. As with Ueno, a method of finding the eyes is described but not a method of adapting the system to a particular vehicle model.
U.S. Pat. No. 5,298,732 to Chen also concentrates in locating the eyes of the driver so as to position a light filter between a light source such as the sun or the lights of an oncoming vehicle, and the driver""s eyes. Chen does not explain in detail how the eyes are located but does supply a calibration system whereby the driver can adjust the filter so that it is at the proper position relative to his or her eyes. Chen references the use of an automatic equipment for determining the location of the eyes but does not describe how this equipment works. In any event, in Chen, there is no mention of monitoring the position of the occupant, other that the eyes, determining the position of the eyes relative to the passenger compartment, or identifying any other object in the vehicle other than the driver""s eyes. Also, there is no mention of the use of a trainable pattern recognition system. A method for finding the eyes is described but not a method of adapting the system to a particular vehicle model.
U.S. Pat. No. 5,305,012 to Faris also describes a system for reducing the glare from the headlights of an oncoming vehicle. Faris locates the eyes of the occupant by using two spaced apart infrared cameras using passive infrared radiation from the eyes of the driver. Faris is only interested in locating the driver""s eyes relative to the sun or oncoming headlights and does not identify, or monitor the occupant or locate the occupant, a rear facing child seat or any other object for that matter, relative to the passenger compartment or the airbag. Also, Faris does not use trainable pattern recognition techniques such as neural networks. Faris, in fact, does not even say how the eyes of the occupant are located but refers the reader to a book entitled Robot Vision (1991) by Berthold Horn, published by MIT Press, Cambridge, Mass. Also, Faris uses the passive infrared radiation rather than illuminating the occupant with ultrasonic or electromagnetic radiation as in some implementations of the instant invention. A method for finding the eyes of the occupant is described but not a method of adapting the system to a particular vehicle model.
The use of neural networks as the pattern recognition technology and the methods of adapting this to a particular vehicle, such as the training methods, is important to this invention since it makes the monitoring system robust, reliable and accurate. The resulting algorithm created by the neural network program is usually only a few hundred lines of code written in the C computer language and is in general fewer lines than when the techniques of the above patents to Ando, Chen and Faris are implemented. As a result, the resulting systems are easy to implement at a low cost making them practical for automotive applications. The cost of the ultrasonic transducers, for example, is expected to be less than about $1 in quantities of one million per year. Similarly, the implementation of the techniques of the above-referenced patents requires expensive microprocessors while the implementation with neural networks and similar trainable pattern recognition technologies permits the use of low cost microprocessors typically costing less than about $5 in quantities of one million per year.
The present invention uses sophisticated software that develops trainable pattern recognition algorithms such as neural networks. Usually the data is preprocessed, as discussed below, using various feature extraction techniques and the results post-processed to improve system accuracy. A non-automotive example of such a pattern recognition system using neural networks on sonar signals is discussed in two papers by Gorman, R. P. and Sejnowski, T. J. xe2x80x9cAnalysis of Hidden Units in a Layered Network Trained to Classify Sonar Targetsxe2x80x9d, Neural Networks, Vol. 1 pp. 75-89, 1988, and xe2x80x9cLearned Classification of Sonar Targets Using a Massively Parallel Networkxe2x80x9d, IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 36, No. 7, July 1988. Examples of feature extraction techniques can be found in U.S. Pat. No. 4,906,940 entitled xe2x80x9cProcess and Apparatus for the Automatic Detection and Extraction of Features in Images and Displaysxe2x80x9d to Green et al. Examples of other more advanced and efficient pattern recognition techniques can be found in U.S. Pat. No. 5,390,136 entitled xe2x80x9cArtificial Neuron and Method of Using Same and U.S. patent application Ser. No. 08/076,601 entitled xe2x80x9cNeural Network and Method of Using Samexe2x80x9d to Wang, S. T. Other examples include U.S. Pat. Nos. 5,235,339 (Morrison et al.), U.S. Pat. No. 5,214,744 (Schweizer et al), U.S. Pat. No. 5,181,254 (Schweizer et al), and U.S. Pat. No. 4,881,270 (Knecht et al). All of the references herein are included herein by reference.
4. Ultrasonics and Optics
Both laser and non-laser optical systems in general are good at determining the location of objects within the two dimensional plane of the image and a pulsed laser radar system in the scanning mode can determine the distance of each part of the image from the receiver by measuring the time of flight through range gating techniques. It is also possible to determine distance with the non-laser system by focusing, or stereographically if two spaced apart receivers are used and, in some cases, the mere location in the field of view can be used to estimate the position relative to the airbag, for example. Finally, a recently developed pulsed quantum well diode laser also provides inexpensive distance measurements as discussed in U.S. provisional patent application Ser. No. 60/114,507, filed Dec. 31, 1999, which is included herein by reference as if the entire contents were copied here.
Acoustic systems are additionally quite effective at distance measurements since the relatively low speed of sound permits simple electronic circuits to be designed and minimal microprocessor capability is required. If a coordinate system is used where the z axis is from the transducer to the occupant, acoustics are good at measuring z dimensions while simple optical systems using a single CCD or CMOS arrays are good at measuring x and y dimensions. The combination of acoustics and optics, therefore, permits all three measurements to be made from one location with low cost components as discussed in commonly assigned U.S. Pat. Nos. 5,845,000 and 5,835,613 cross-referenced above.
One example of a system using these ideas is an optical system which floods the passenger seat with infrared light coupled with a lens and a receiver array, e.g., CCD or CMOS array, which receives and displays the reflected light and an analog to digital converter (ADC) which digitizes the output of the CCD or CMOS and feeds it to an Artificial Neural Network (ANN) or other pattern recognition system for analysis. This system uses an ultrasonic transmitter and receiver for measuring the distances to the objects located in the passenger seat. The receiving transducer feeds its data into an ADC and from there the converted data is directed into the ANN. The same ANN can be used for both systems thereby providing full three-dimensional data for the ANN to analyze. This system, using low cost components, will permit accurate identification and distance measurements not possible by either system acting alone. If a phased array system is added to the acoustic part of the system, the optical part can determine the location of the driver""s ears, for example, and the phased array can direct a narrow beam to the location and determine the distance to the occupant""s ears.
Although the use of ultrasound for distance measurement has many advantages, it also has some drawbacks. First, the speed of sound limits the rate at which the position of the occupant can be updated to approximately 10 milliseconds, which though sufficient for most cases, is marginal if the position of the occupant is to be tracked during a vehicle crash. Second, ultrasound waves are diffracted by changes in air density that can occur when the heater or air conditioner is operated or when there is a high-speed flow of air past the transducer. Third, the resolution of ultrasound is limited by its wavelength and by the transducers, which are high Q tuned devices. Typically, the resolution of ultrasound is on the order of about 2 to 3 inches. Finally, the fields from ultrasonic transducers are difficult to control so that reflections from unwanted objects or surfaces add noise to the data.
Ultrasonics alone can be used in several configurations for monitoring the interior of a passenger compartment of an automobile as described in the above-referenced patents and patent applications and in particular in U.S. patent application Ser. No. 08/798,029. Using the teachings of this invention, the optimum number and location of the ultrasonic and/or optical transducers can be determined as part of the adaptation process for a particular vehicle model.
In the cases of the instant invention, as discussed in more detail below, regardless of the number of transducers used, a trained pattern recognition system, as defined above, is used to identify and classify, and in some cases to locate the illuminated object and its constituent parts.
5. Applications
The applications for this technology are numerous as described in the patents and patent applications listed above. However, the main focus of the instant invention is the process of adapting the system in the patents and patent applications referenced above for the detection of the presence of an occupied child seat in the rear facing position or an out-of-position occupant and the detection of an occupant in a normal seating position. The system is designed so that in the former two cases, deployment of the occupant protection apparatus (airbag) may be controlled and possibly suppressed and in the latter, it will be controlled and enabled.
One preferred implementation of a first generation occupant sensing system, which is adapted to various vehicle models using the teachings presented herein, is an ultrasonic occupant position sensor. This system uses an Artificial Neural Network (ANN) to recognize patterns that it has been trained to identify as either airbag enable or airbag disable conditions. The pattern is obtained from four ultrasonic transducers that cover the front passenger seating area. This pattern consists of the ultrasonic echoes bouncing off of the objects in the passenger seat area. The signal from each of the four transducers consists of the electrical image of the return echoes, which is processed by the electronics. The electronic processing comprises amplification, logarithmic compression, rectification, and demodulation (band pass filtering), followed by discretization (sampling) and digitization of the signal. The only software processing required, before this signal can be fed into the artificial neural network, is normalization (i.e., mapping the input to numbers between 0 and 1). Although this is a fair amount of processing, the resulting signal is still considered xe2x80x9crawxe2x80x9d, because all information is treated equally.
In general, it is an object of the present invention to provide new and improved arrangements for detecting the presence of an object in a passenger compartment of a vehicle.
It is another object of the present invention to provide a new and improved method for developing a system for identifying the presence, position and orientation of an object in a vehicle.
It is another broad object of the present invention to provide a method for developing a system for accurately detecting the presence of an occupied rear-facing child seat in order to prevent an occupant protection apparatus such as an airbag from deploying, when the airbag would impact against the rear-facing child seat if deployed.
It is yet another broad object of the present invention to provide a method for developing a system for accurately detecting the presence of an out-of-position occupant in order to prevent one or more deployable occupant protection apparatus such as airbags from deploying when the airbag(s) would impact against the head or chest of the occupant during its initial deployment phase causing injury or possible death to the occupant.
The invention is also a method to develop and adapt a system to identify, locate and monitor occupants, including their parts, and other objects in the passenger compartment and in particular an occupied child seat in the rear facing position or an out-of-position occupant, by illuminating the contents of the vehicle with ultrasonic or electromagnetic radiation, for example, by transmitting radiation waves from a wave generating apparatus into a space above the seat, and receiving radiation modified by passing through at least part of the space above the seat using two or more transducers properly located in the vehicle passenger compartment, in specific predetermined optimum locations. More particularly, this invention relates to a method for appropriately locating and mounting the transducers and for analyzing the received radiation from any object which modifies the waves, in order to achieve an accuracy of recognition heretofore not possible. Outputs from the receivers, are analyzed by appropriate computational means employing trained pattern recognition technologies, to classify, identify and/or locate the contents, and/or determine the orientation of, for example, a rear facing child seat. In general, the information obtained by the identification and monitoring system is used to affect the operation of some other system, component or device in the vehicle and particularly the passenger and/or driver airbag systems, which may include a front airbag, a side airbag, a knee bolster, or combinations of the same. However, the information obtained can be used for a multitude of other vehicle systems.
When the vehicle interior monitoring system developed using the teachings of this invention is installed in the passenger compartment of an automotive vehicle equipped with a occupant protection apparatus, such as an inflatable airbag, and the vehicle is subjected to a crash of sufficient severity that the crash sensor has determined that the protection apparatus is to be deployed, the system, developed in accordance with the invention, has determined prior to the deployment whether a child placed in the rear facing position in the child seat is present and if so, a signal has been sent to the control circuitry that the airbag should be controlled and most likely disabled and not deployed in the crash. It must be understood though that instead of suppressing deployment, it is possible that the deployment may be controlled so that it might provide some meaningful protection for the occupied rear-facing child seat. The system developed using the teachings of this invention also determines the position of the vehicle occupant relative to the airbag and controls and possibly disables deployment of the airbag if the occupant is positioned so that he/she is likely to be injured by the deployment of the airbag. As before, the deployment is not necessarily disabled but may be controlled to provide protection for the out-of-position occupant.
Principle objects and advantages of the methods in accordance with the invention are:
1. To provide a reliable method for developing and adapting a system for recognizing the presence and orientation of a child seat on a particular seat of a motor vehicle.
2. To provide a reliable method for developing and adapting a system for recognizing the presence of a human being on a particular seat of a motor vehicle.
3. To provide a reliable method for developing and adapting a system for determining the position, velocity or size of an occupant in a motor vehicle.
4. To provide a reliable method for developing and adapting a system for determining in a timely manner that an occupant is out-of-position, or will become out-of-position, and likely to be injured by a deploying airbag.
5. To provide a method for locating transducers within the passenger compartment at specific locations such that a high reliability of classification of objects and their position is obtained, from the signals generated by the transducers.
6. To provide a method for combining a variety of transducers including seatbelt payout sensors, seatbelt buckle sensors, seat position sensors, seatback position sensors, and weight sensors into a system and adapt that system so as to provide a highly reliable occupant presence and position system when used in combination with electromagnetic, ultrasonic or other radiation sensors.
7. To provide methods for controlling deployment of an occupant restraint, optionally based on a determined position of the occupant.
8. To provide methods for determining whether an object is a child seat, forward or rearward facing, and optionally control deployment of an occupant restraint device based on such determination.
In order to achieve some of the objects set forth above, arrangements in accordance with the invention includes a plurality of ultrasonic transmitters for transmitting ultrasonic waves into the passenger compartment, each operating at a distinct transmitting frequency and positioned at a distinct location relative to the other transmitter(s), at least one receiver disposed so as to receive from the passenger compartment ultrasonic waves transmitted from a transmitter and modified by passing through at least part of the passenger compartment, and a processor operatively coupled to the receiver(s). Based on the ultrasonic waves received by the receiver(s), the processor determines whether an object is located in the passenger compartment, determines the location of the object and/or controls deployment of a safety restraint device. It is not required that the processor make a separate determination that an object is present before determining its location or to deploy the safety restraint device. Rather, the processor can make a determination directly as to whether to deploy the restraint device based on the ultrasonic waves without making an initial determination that an occupant is present. The determination by the processor may entail the use of pattern recognition techniques, such as a neural network or sensor fusion.
In more specific embodiments, each receiver and transmitter may be arranged to form a transducer. One transmitter (or transducer) may be arranged on a ceiling of the vehicle and a second transmitter (or transducer) arranged at a different location in the vehicle, e.g., on the dashboard, such that a first axis connecting the first and second transmitters is substantially parallel to a second axis traversing a volume in a passenger compartment of the vehicle above a seat in which an object is situated. A third transmitter (or transducer) may be provided on or adjacent an interior side surface of the passenger compartment. A fourth transmitter (or transducer) may be provided on or adjacent an interior side surface of the passenger compartment.
Other arrangements include a first receiver arranged on a ceiling of the vehicle, a second receiver arranged at a different location in the vehicle than the first receiver such that a first axis connecting the first and second receivers is substantially parallel to a second axis traversing a volume in a passenger compartment of the vehicle above a seat in which an object is situated, and a third receiver arranged at a different location in the passenger compartment than the first and second receivers. Each receiver comprising distance measurement means such that a first distance from the first receiver to the object is obtained based on the output of the first receiver; a second distance from the second receiver to the object is obtained based on the output of the second receiver and a third distance from the third receiver to the object is obtained based on the output of the third receiver. A processor analyzes these distances and determines if an object is present based thereon, determines the location of the object and/or controls deployment of a safety restraint device. The processor does not have to determine whether an object is present prior to formulating the control signal for the restraint device but rather can make a direct determination as to the manner in which the restraint device will be controlled based on the distances as obtained from the receivers.
In other, more specific embodiments, the receivers are arranged to receive ultrasonic or electromagnetic radiation and may all be of the same type A fourth receiver may be arranged at a different location in the passenger compartment than the first, second and third receivers and comprise distance measurement means. The processor analyzes the distance between the fourth receiver and the object when determining if an object is present, determining the location of the object or controlling of the restraint device. In the event that the receivers receive, e.g., ultrasonic waves, then one or more transmitters is/are provided for transmitting waves into the passenger compartment.
Another embodiment of a method for controlling deployment of an occupant restraint device in a vehicle comprises the steps of arranging a first ultrasonic transducer on a ceiling of the vehicle and a second ultrasonic transducer at a different location in the vehicle (e.g., on the dashboard) such that a first axis connecting the first and second transducers is substantially parallel to a second axis traversing a volume in a passenger compartment of the vehicle above a seat in which an object is situated, transmitting ultrasonic waves from the first transducer into the passenger compartment, receiving ultrasonic waves reflected off the object in the passenger compartment at the first transducer and calculating a first distance from the first transducer to the object based on the time difference between the transmitted waves when transmitted from the first transducer and the reflected waves when received at the first transducer. In a similar manner, different ultrasonic waves are transmitted from the second transducer into the passenger compartment, ultrasonic waves reflected off the object in the passenger compartment are received at the second transducer and a second distance from the second transducer to the object is calculated based on the time difference between the transmitted waves when transmitted from the second transducer and the reflected waves when received at the second transducer. Deployment of the occupant restraint device and/or a determination of whether the object is a child seat is/are made based on the first distance and the second distance.
Control of deployment of the occupant restraint device may entail suppressing deployment of the occupant restraint device or if the restraint device includes an airbag, controlling a rate of generation of a gas used to inflate the airbag and/or an amount of gas generated for inflating the airbag.
A third transducer may be arranged, e.g., on or adjacent an interior side surface of the passenger compartment, to transmit different ultrasonic waves into the passenger compartment and receive ultrasonic waves reflected off the object in the passenger compartment. The distance from the third transducer to the object is calculated based on the time difference between the transmitted waves when transmitted from the third transducer and the reflected waves when received at the third transducer. This distance is used in consideration of the control of the deployment of the airbag and/or the determination as to whether the object is a child seat. A fourth transducer may also be arranged to transmit different ultrasonic waves into the passenger compartment and receive ultrasonic waves reflected off the object in the passenger compartment. A fourth distance from the fourth transducer to the object is calculated based on the time difference between the transmitted waves when transmitted from the fourth transducer and the reflected waves when received at the fourth transducer. This distance is also used in consideration of the control of the deployment of the airbag and/or the determination as to whether the object is a child seat.
Another method for controlling deployment of an occupant restraint device in a vehicle and/or determining whether an object in a seat is a child seat comprises the steps of arranging a first receiver on a ceiling of the vehicle and a second receiver at a different location in the vehicle such that a first axis connecting the first and second receivers is substantially parallel to a second axis traversing a volume in a passenger compartment of the vehicle above a seat in which an object is situated, mounting a third receiver at a different location in the passenger compartment than the first and second receivers, each receiver comprising distance measurement means, calculating a first distance from the first receiver to the object based on the output of the first receiver, calculating a second distance from the second receiver to the object based on the output of the second receiver, and calculating a third distance from the third receiver to the object based on the output of the third receiver. Deployment of the occupant restraint device and/or the determination as to whether the object is a child seat is/are made based on the first distance, the second distance and the third distance. The receivers may be constructed to receive ultrasonic radiation or electromagnetic radiation and may all be of the same type.
Another method for controlling deployment of an occupant restraint device in a vehicle and/or determining whether an object in a seat is a child seat comprises the steps of transmitting ultrasonic waves from a first transducer into a passenger compartment of the vehicle, receiving waves reflected off an object in the passenger compartment at the first transducer, calculating a first distance from the first transducer to the object based on the time difference between the transmitted waves when transmitted from the first transducer and the reflected waves when received by the first transducer, transmitting different ultrasonic waves from a second transducer into the passenger compartment, receiving waves reflected off the object in the passenger compartment at the second transducer and calculating a second distance from the second transducer to the object based on the time difference between the transmitted waves when transmitted by the second transducer and the reflected waves when received by the second transducer. Deployment of the occupant restraint device and/or the determination as to whether the object is a child seat is/are made based on the first distance and the second distance. The distance calculation steps comprise the step of applying an algorithm generated by means of a pattern recognition algorithm generating program (e.g., a neural network computer program) based on the time distribution of the echo pattern of the reflected waves in order to determine the distance from the respective transducer to the object.
In one embodiment of a method of developing a system for determining the occupancy state of a seat in a passenger compartment of a vehicle comprises the steps of mounting transducers in the vehicle, which transducers would be affected by the occupancy state of the seat, forming at least one database comprising multiple data sets, each data set representing a different occupancy state of the seat and being formed by receiving data from the transducers while the seat is in that occupancy state, and processing the data received from the transducers, and creating a first algorithm from the database(s) capable of producing an output indicative of the occupancy state of the seat upon inputting a new data set representing an occupancy state of the seat. The new data set would be formed, e.g., during use of the vehicle after the algorithm is installed in the control circuitry of the vehicle. The first algorithm may be created by inputting the database(s) into an algorithm-generating program, and running the algorithm-generating program to produce the first algorithm. The first algorithm could be a neural network algorithm, in which case, the back propagation method could be used when generating the neural network algorithm.
The occupancy states of the seat include occupancy of the seat by an object selected from the group comprising occupied and unoccupied rear facing infant seats, forward facing humans, out-of-position humans, occupied and unoccupied forward facing child seats and empty seats. The occupancy states of the seat should also include occupancy by the objects in multiple orientations and/or having at least one accessory selected from a non-exclusive group comprising newspapers, books, maps, bottles, toys, hats, coats, boxes, bags and blankets.
The data can be pre-processed prior to being formed into the data sets. This may entail using data created from features of the data in the data set, which features might be selected from a group comprising the normalization factor, the number of data points prior to a peak, the total number of peaks, and the mean or variance of the data set. Also, the data sets could be mathematically transformed using normalization, truncation, logarithmic transformation, sigmoid transformation, thresholding, averaging the data over time, Fourier transforms and/or wavelet transforms. Further, pre-processing could entail subtracting data in one data set from the corresponding data in another data set to create a third data set of differential data.
The processing step may comprise the step of converting the analog data from the transducers to digital data and combining the digital data from a plurality of the transducers to form a vector comprising a string of data from each of the transducers. As such, the first algorithm is created such that upon inputting a vector from a new data set will produce an output representing the occupancy state of the vehicle seat. The vectors in the database can be normalized so that all values of the data that comprise each vector are between a maximum and a minimum.
Another disclosed method of developing a system for determining the occupancy state of the vehicle seat in the passenger compartment of a vehicle comprises the steps of forming data sets by obtaining data representative of various occupying objects at various positions in the passenger compartment and operating on at least a portion of the data to reduce the magnitude of the largest data values in a data set relative to the smallest data values, forming a database comprising multiple data sets, and creating an algorithm from the database capable of producing an output indicative of the occupancy state of the vehicle seat upon inputting a data set representing an occupancy state of the seat. Operating on the data may entail using an approximate logarithmic transformation function.
A disclosed method of developing a database for use in developing a system for determining the occupancy state of a vehicle seat in accordance with the invention comprises the steps of mounting transducers in the vehicle and which would be affected by the occupancy state of the seat, providing the seat with an initial occupancy state, receiving data from the transducers, processing the data from the transducers to form a data set representative of the initial occupancy state of the vehicle seat, changing the occupancy state of the seat and repeating the data collection process to form another data set, collecting at least 1000 data sets into a first database, each representing a different occupancy state of the seat and creating an algorithm from the first database which correctly identifies the occupancy state of the seat for most of the data sets in the first database. The algorithm is tested using a second database of data sets which were not used in the creation of the algorithm. The occupancy states in the second database are which were not correctly identified by the algorithm are identified and new data comprising similar occupancy states to the incorrectly identified states is collected. The new data is combined with the first database, a new algorithm is created based on the combined database and this process is repeated until the desired accuracy of the algorithm is achieved.
Another disclosed method of developing a system for determining the occupancy state of a passenger compartment seat of a vehicle comprises the steps of mounting a plurality of ultrasonic transducers in the vehicle (which transducers would be affected by the occupancy state of the seat), receiving an analog signal from each of the transducers, processing the analog signals from the transducers to form a data set comprising multiple data values from each transducer representative of the occupancy state of the vehicle the data processing comprising the steps of demodulation, sampling and digitizing of the transducer data to create a data set of digital data forming a database comprising multiple data sets and creating at least one algorithm from the database capable of producing an output indicative of the occupancy state of the seat upon inputting a new data set representing an occupancy state of the seat.
Still another disclosed method of developing a system for determining the occupancy state of a vehicle seat in a passenger compartment of a vehicle comprises the steps of mounting a set of transducers on the vehicle, receiving data from the transducers, processing the data from transducers to form a data set representative of the occupancy state of the vehicle, forming a database comprising multiple data sets, creating an algorithm from the database capable of producing an output indicative of the occupancy state of the vehicle seat upon inputting a new data set, and developing a measure of system accuracy. At least one transducer is removed from the transducer set, a new database is created containing data only from the reduced number of transducers, a new algorithm is developed based on the new database and tested to determine the new system accuracy. The process of removing transducers, algorithm development and testing is continued until the minimum number of sensors is determined which produces an algorithm having desired accuracy. The transducers are selected from a group consisting of ultrasonic transducers, optical sensors, capacitive sensors, weight sensors, seat position sensors, seatback position sensors, seat belt buckle sensors, seatbelt payout sensors, infrared sensors, inductive sensors and radar sensors.
Yet another disclosed method of developing a system for determining the occupancy state of the driver and passenger seats of a vehicle comprises the steps of mounting ultrasonic transducers having different transmitting and receiving frequencies in a vehicle such that transducers having adjacent frequencies are not within the direct ultrasonic field of each other, receiving data from the transducers, processing the data from the transducers to form a data set representative of the occupancy state of the vehicle, forming at least one database comprising multiple data sets and creating at least one algorithm from the at least one database capable of producing an output indicative of the occupancy state of a vehicle seat upon inputting a new data set.
These and other objects and advantages will become apparent from the following description of the preferred embodiments of the vehicle identification and monitoring system of this invention.